Black-box testing using flowgraphs: an experimental assessment of effectiveness and automation potential

A black-box testing strategy based on Zweben et al.’s specification-based test data adequacy criteria is explored. The approach focuses on generating a flowgraph from a component’s specification and applying analogues of white-box strategies to it. An experimental assessment of the fault-detecting ability of test sets generated using this approach was performed for three of Zweben’s criteria using mutation analysis. By using precondition, postcondition, and invariant checking wrappers around the component under test, fault detection ratios competitive with white-box techniques were achieved. Experience with a prototype test set generator used in the experiment suggests that practical automation may be feasible.